Independence and Bayesian Updating Methods

Duda, Hart, and Nilsson [ 1] have set forth a method for rule-based inference systems to use in updating the probabilities of hypotheses on the basis of multiple items of new evidence. Pednault, Zucker, and Muresan [2] claimed to give condi­ tions under which independence assumptions made by Duda et al. preclude updating-that is, prevent the evidence from altering the probabilities of the hypotheses. Glymour [3] refutes Pednault et al.'s claim with a counterexample of a rather special form (one item of evidence is incompatible with all but one of the hypotheses); he raises, but leaves open, the question whether their result would be true with an added assumption to rule out such special cases. We show that their result does not hold even with the added assumption, but that it can nevertheless be largely salvaged. Namely, under the conditions assumed by Pednault et al., at most one of the items of evidence can alter the probability of any given hypothesis; thus, although updating is possible, multiple updating for any of the hypotheses is precluded. BACKGROUND Duda, Hart, and Nilsson [1] consider the problem of updating the probability of a hypothesis H w�th prior probability P (H) when new evidence is obtained in the form of proposi­ tions Ei for which the conditional probabilities P (E,. I H) and P (Ei I li) are known. They assume that the E,are conditionally independent, both on condition H and on condition H, so that m P (E 1 • • 'Em I H)= II P (E,I H) i=1 m P(E1 ·''Em I H)= II P(Ei I H). i=1 They can then write an updating formula for the odds on H in terms of a product of likelihood ratios: 28 P(H I E1' ·'Em) = P(H) IT P(Ei I H) P(Ji I E1' ··Em) P(Ji) i=1 P(E,I H) . Pednault, Zucker, and Muresan [2], in analyzing this updating scheme, considered the consequences of imposing the independence assumptions for each hypothesis Hi of a jointly exhaustive, mutually exclusive set. They [2] and other writers (see [3] and other references therein) agree that the assumptions are unreasonably strong, but there has been some confusion over the exact extent of the undesirable consequences. Pednault et al. [2] concluded that if there were at least three hypotheses, then no updating could take place-that the assumptions are too strong to be satisfied unless p (E,. I Hi) = p (E,. I Hi) = p (E,.) . holds for all i and j , and consequently . P (Hi I E 1' · · Em) = P (Hi). However, Glymour [3] gives a counterexample to their conclusion-three jointly exhaustive, mutu­ ally exclusive hypotheses � and two evidence propositions Ei that satisfy the independence assumptions but allow updating to occur. He points out that Pednault et al. had relied on an erroneous result claimed by Hussain [4], also refuted by his counterexample. Glymour notes that the evidence proposition E 2 ·of his counterex­ ample has the special property that P (E 2 I H 2) = P(E2I H3) = 0, so that E2 determines a posterior probability of 1 for one hypothesis, H 17 and 0 for the rest. He raises, and leaves open, the question whether Pednault et al. 's result would be true with the additional requirement that for all i , (1) In the next section we answer that question by giving a counterexample that satisfies ( 1 ). Glymour's counterexample has another spe­ cial property, one that is sufficient to make it a valid _ counterexample: P (E 1 I Hi)= P (E 1 I Hi) = P (E 1), so that E 1 produces no updating; only E2 produces updating. We give a I